A system-level neural model of the brain mechanisms underlying instrumental devaluation in rats

Goal-directed behaviours are defined by the presence of two kinds of effect on instrumental learning. First, degrading the contingencies between produced actions and desired outcomes diminishes the number of instrumental responses; second, devaluing a reward results in a lower production of instrumental actions to obtain it. We present a computational model of the neural processes underlying instrumental devaluation in rats. The model reproduces the interaction between the basolateral complex of the amygdala (BLA) and the limbic, associative and somatosensory striato-cortical loops. Firing-rate units are used to abstract the activity features of neural populations. Learning is reproduced through the use of dopamine-dependent simple and differential hebbian rules. Constraints from anatomy of the projections between neural systems are taken into account. The central hypothesis implemented in the model is that pavlovian associations learned within the BLA between manipulanda and rewards modulate goal selection through the activation of the nucleus accumbens core (NaccCo). Selection processes happening in the limbic basal ganglia with the activation of the NaccCo decide which outcome is choosen as a goal within the prelimbic cortex (PL). Connections between the BLA and the NaccCo are learned through hebbian associations mediated by feedbacks from the PL to the NaccCo. Information about selected goals from the limbic striato-cortical loop influences action selection in the sensorimotor loop both through cortico-cortical projections and through a striato-nigro-striatal dopaminergic pathway passing through the associative striato-cortical loop. The model is tested as part of the control system of a simulated rat. Instrumental devaluation tasks are reproduced. Simulated lesions of the BLA, the NaccCo, the PL and the dorsomedial striatum (DMS) both before and after training reproduce the behavioural effect of lesions in real rats. The model provides predictions about the effects of still undocumented lesions

Selection of cortical dynamics for motor behaviour by the basal ganglia

The basal ganglia and cortex are strongly implicated in the control of motor preparation and execution. Re-entrant loops between these two brain areas are thought to determine the selection of motor repertoires for instrumental action. The nature of neural encoding and processing in the motor cortex as well as the way in which selection by the basal ganglia acts on them is currently debated. The classic view of the motor cortex implementing a direct mapping of information from perception to muscular responses is challenged by proposals viewing it as a set of dynamical systems controlling muscles. Consequently, the common idea that a competition between relatively segregated cortico-striato-nigro-thalamo-cortical channels selects patterns of activity in the motor cortex is no more suf?cient to explain how action selection works. Here, we contribute to develop the dynamical view of the basal ganglia-cortical system by proposing a computational model in which a thalamo-cortical dynamical neural reservoir is modulated by disinhibitory selection of the basal ganglia guided by top-down information, so that it responds with different dynamics to the same bottom-up input. The model shows how different motor trajectories can so be produced by controlling the same set of joint actuators. Furthermore, the model shows how the basal ganglia might modulate cortical dynamics by preserving coarse-grained spatiotemporal information throughout cortico-cortical pathways

Possible roles of orbitofrontal-amygdala loop in modulation of basal ganglia activation

In the last years research on emotion focused on the role of amygdala-orbitofrontal loops in learning associations between internal states of the organism and external context, and in modulating choice behaviour. Amygdala is now thought to participate actively in current reward value-response learning Furthermore, amygdala seems to act as an effector of choice behaviour elaboration in orbitofrontal cortex. The influence of amygdala-orbitofrontal loop in reward behaviours and choice behaviours probably passes through its output connections to corticostriatal loop (input from basolateral nuclei of amygdala to nucleus accumbens). At the same time, amygdala modulates hypothalamus and brain stem arousal and controls such systems through pathways mainly running from the amygdala?s central nucleus to brain stem. Habit learning does not require amygdala?s processing. Overall, these findings gives evidence of the existence of two different systems involved in reward-based behaviour, one more dynamic and related to the mapping between the actual context and inner states, and the other, more stable and acquired over long periods of time, relatively independent of actual inners states

The interaction between basal ganglia and cortex during action learning and selection: a computational model

No abstract availabl

Learning and selecting actions: a computational model of the basal-ganglia cortical dynamic interplay

No abstract availabl

The hierarchical organisation of cortical and basal-ganglia systems: a computationally-informed review and integrated hypothesis

To suitably adapt to the challenges posed by reproduction and survival, animals need to learn to select when to perform different behaviours, to have internal criteria for guiding these learning processes, and to perform behaviours efficiently once selected. To implement these processes, their brain must be organised in a suitable hierarchical fashion. Here we briefly review two types of neural/behavioural/computational literatures, focussed respectively on cortex and on sub-cortical areas, and highlight their important limitations. Then we review two computational modelling works of the authors that exemplify the problems, brain areas, experiments, main concepts and limitations of the two research threads. Finally we propose a theoretical integration of the two views, showing how this allows to solve most of the problems found by the two accounts if taken in isolation. The overall picture that emerges is that the cortical and the basal ganglia systems form two highly-organised hierarchical systems working in close synergy and jointly solving all the challenges of choice, selection, and implementation needed to acquire and express adaptive behaviour

A computational model of goal-driven behaviours and habits in rats

No abstract availabl

Computational principles underlying the functioning of amygdala in the affective regulation of behaviour

This paper presents a short review, compiled with a computational perspective, of the empirical neuroscientific evidence related to amygdala, a brain complex situated at the core of various brain systems underlying motivations and emotions. The functions of amygdala are fundamental for organisms\u27 adaptive behaviour as they allow them to assign subjective saliency and value to experienced world states, so enhancing the adaptive power of their cognitive processes. In this respect, the major goal of the review is outlining the main computational functionalities of amygdala emerging from the neuroscientific investigations on affective processes so as to contribute to highlight the general architectural and functioning mechanisms underlying organisms\u27 emotional processes. This effort is also expected to fertilise the design of robot controllers exhibiting a flexibility and autonomy comparable to that of real organisms

Distal place recognition based navigation control inspired by hippocampus-amygdala interactions

We present a novel robot navigation system based on distal place and value recognition. The navigation control system is inspired by the hippocampus - amygdala circuit that is involved in place learning/recognition and stimulus value association